Recently, I shared how I have become very popular recently because Google has withdrawn Google Site Search. Many large companies who were using it as their website search engine recoiled in horror at the prospect of having ads on their site search results page—ads from their competitors. Try explaining that to the boss.
So if you are one of the many that is wondering just how to replace your site search engine, these are the basic steps:
- Do your business case. Maybe “there will be ads” is enough, but I often help clients show the revenue improvements and cost savings that stem from making the search results better. (Yes, you can project them and yes you can measure that they happened after the fact.)
- Gather your requirements. How many pages do you have? How many searches? How many countries and languages? Do you crawl the pages or push from the CMS? Do you curate some results manually to make sure they are good? Cloud or on-premises? Speed? Price?
- Round up the suspects. Everyone should look at Solr and Elasticsearch—they are free open source and do a very good job in many situations, but there are plenty of proprietary engines out there than can make a ton of sense.
- Run some trials. See if you can take a test drive. Ask some others for their experience. See how it seems to work.
- Install, test, and cut over. Standard IT stuff, right? Wrong.
You probably wouldn’t randomly change pages on your site, hoping that conversion is improving. You test them. You use A/B or multivariate testing. Don’t you need to do the same thing for your search engine?
You set up the new search engine (you can even set up several if you are ambitious) and you get the pages indexed and start tweaking the settings. (Consultants like me can help with this if you don’t know how.)
And then you siphon a small percentage of the production searches to the new search engine(s). You can run them from test servers. Just make sure the load is low enough that they perform well and that if they keel over that you fall back over to the regular production search engine.
Why do this? Because what you really want to test is whether your searchers are finding what they are looking for. You can use a site search analytics tool, such as SoloSegment Site Search Inspector [Full disclosure: I am their Senior Strategist] to check whether searches are successful or not.
What makes a search successful? That search gets search results (it isn’t a “not found”), at least one of those results gets clicked, and the searcher doesn’t “pogostick” back to the search results screen within a few seconds. If they get no results or they don’t click on anything or they pogostick back, you know that was bad. You don’t know that the successes are good, but it’s the way to bad. Eliminating the bad leaves mostly good.
So, clients use Site Search Inspector to constantly monitor their searches and improve the number of successes, where searchers presumably found what they were looking for. But when you are migrating to a new search engine, that’s when you really need it.
Let’s say that you are using Google Site Search now. And maybe it isn’t the greatest search engine ever, but it is working good enough that you don’t get a lot of complaints. What happens when you put in the new one? Most search engines have the potential to deliver better results than Google Site search, but they also need a lot more care and feeding. They need their algorithms tweaked, their defaults messed with—they don’t always work so well out of the box. If you just install it, test that it operates, and cut over, you might be tragically harming your business results and have no way to quickly fix it. Conversion rates drop. Call center costs go up. All because the new search engine isn’t as good out of the box as Google Site Search.
So you can use SoloSegment Site Search Inspector to test the new search engine. Install SoloSegment Keyword Inspector to get a baseline of analytics for your current Google Site Search experience. Set up the new search engine to get (say) 1% of the production searches, but continue to route most of the searches through your existing Google Site Search engine.
Make a change to your search application so that every Google Site Search request has a new URL parameter (&engine=google), but route a small, random sample of searches through your new search engine (&engine=elastic). You can even test several at once. Site Search Inspector can show you the numbers for each search engine, so you can see how they compare. Take a look at the no results searches, at the clickthrough rate, and the pogostick rate. How do the two engines compare for the same exact search keywords?
Now, you suddenly have the information you need to know exactly when your new search engine is good enough to replace (or even surpass) your old search engine. No guesswork involved. Tune your new search engine until it surpasses the old search engine, and then install it on big production servers and route all searches to the new search engine.
You don’t have to do this. You can just cut over and cross your fingers that no one will notice. But if you do, you might have weeks or months of work to fix the new engine and you are losing money every day and getting screamed at every day. Instead, why not use the old A/B test and be confident that you have it working from day one?
